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EXECUTIVE  SUMMARY 


Background 

In  1992,  the  American  Association  of  Dental  Schools  (AADS)  issued  a  resolution  calling 
for  the  development  of  programs  that,  by  the  year  2000,  would  require  all  U.S.  dental  school 
graduates  to  enroll  in  a  one-year  postdoctoral  triuning  program.  If  the  resolution  to  require 
postdoctoral  Advanced  Education  in  General  Dentistry  (AEGD)  is  adopted,  the  Army  will  be 
forced  to  choose  between  continuing  its  existing  programs  or  acquiring  only  civilian-trained 
dentists.  Budgetary  constraints,  the  value  of  the  existing  programs  to  the  military,  and 
consequences  of  the  choice  must  alt  be  considered. 

Recently,  concerns  have  been  raised  that  not  enough  qualified  applicants  have  applied  for 
the  Army’s  AEGD  program  and,  consequently,  accessions  from  the  lower  half  of  dental  school 
classes  have  had  to  be  selected  to  fill  existing  positions.  Program  mentors  have  also  expressed 
concern  that  recent  participants  have  not  been  performing  as  well  as  past  participants, 
presumably  because  of  their  lower  academic  qualifications. 

As  part  of  the  Army  Medical  Department  Studies  Program,  the  Dental  Studies  Division 
of  the  Health  Care  Studies  and  Clinical  Investigation  Activity  initiated  a  study  to  examine  factors 
associated  with  success  in  the  Army-sponsored  AEGD  program  and  to  determine  the  likelihood 
that  new  accessions  with  lower  dental  school  class  standings  would  succeed  in  the  AEGD 
program. 

This  report  provides  the  Army  Dental  Care  System  leadership  with  a  framework  for 
evaluating  the  existing  AEGD  selection  process. 
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Methods 

This  study  has  two  components:  !)  the  creation  of  a  measure  of  the  dimensions  of 
performance  during  the  AEGD,  i.e.  an  index  of  program  success,  and  2)  the  identification  of 
background  factors  that  are  valid  predictors  of  success  in  the  AEGD  program. 

Academic  Evaluation  Reports  for  25  recent  AEGD  graduates  were  analyzed  for  content 
by  two  experienced  dental  officers.  The  important  dimensions  of  the  AEGD  program  were 
established,  and  rating  principles  and  scales  were  developed  for  reviewers  to  use  when  evaluating 
a  participants’  Academic  Evaluation  Report.  This  resulted  in  1)  definitions  of  the  important 
dimensions  of  the  AEGD  program;  2)  summary  principles  that  highlight  characteristics  to  look 
for  when  determining  the  level  of  achievement  demonstrated  within  the  content  of  the  evaluation; 
and  3)  numerical  equivalents  that  allow  the  accomplishments  within  each  dimension  to  be 
translated  into  quantitative  indices  of  success.  The  reliabilities  of  the  reviewers’  ratings  were 
estimated  using  the  kappa  statistic  and  percent  absolute  agreement. 

Multivariate  regression  was  applied  to  evaluate  biographical  factors  from  90  recent  AEGD 
graduates  for  their  association  with  the  created  Program  Success  Index.  Logistic  regression 
assessed  the  usefulness  of  the  background  factors  in  classifying  individuals  as  above  average  or 
below  average  in  terms  of  success  in  the  AEGD  program.  Additional  analyzes  examined  the 
effect  that  dental  school  class  standing  had  on  success  in  the  AEGD  program  and  subsequent 
success  within  the  Army  Dental  Corps. 
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Results 


Consensus  of  the  reviewers  determined  the  major  dimensions  of  successful  performance 
in  the  AEGD  program;  these  were  then  utilized  to  create  a  Program  Success  Index.  Background 
factors  available  to  selection  panels  were  related  to  the  Program  Success  Index;  prior  military 
education,  dental  school  class  standing,  graduation  age,  a  Regular  Army  appointment,  and  an 
interaction  term  (prior  military  education  with  age  at  graduation)  emerged  as  significant 
predictors  of  success  in  the  AEGD  program.  These  factors  accounted  for  approximately  21 
percent  of  the  variation  in  Program  Success  Index  scores. 

In  a  logistic  regression  model,  the  identified  predictors  correctly  classified  67.8  percent 
of  the  AEGD  participants  as  either  above  or  below  average  in  terms  of  success  in  the  program. 
And,  in  this  model,  AEGD  participants  from  the  top  half  of  their  dental  school  class  were  1 .55 
times  as  likely  to  be  classified  as  above  average  in  terms  of  program  success  than  were  those 
from  the  bottom  half. 

Class  standing  by  itself  did  not  effect  the  Program  Success  Index  score  or  subsequent 
success  in  an  Army  career  as  measured  by  Officer  Evaluation  Reports,  promotion  to  the  rank 
of  major,  or  sel«:tion  for  advanced  dental  specialty  education. 

Conclusion 

The  primary  value  of  the  approach  used  in  this  study  is  in  helping  to  further  clarify  the 
nature  of  success  in  AEGD  through  the  development  of  the  Program  Success  Index.  This  study 
suggests  that  the  dimensions  of  success  in  AEGD  uncovered  in  the  analysis  of  the  Academic 
Evaluation  Report  can  be  reliably  quantified  by  experienced  reviewers  and  can  serve  as  a 
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framework  for  establishing  evaluative  criteria  for  AEGD  programs.  The  results  suggest  that 
while  background  factors  used  by  selection  committees  can  distinguish  between  levels  of  success 
in  an  AEGD  program,  other  factors  should  be  considered.  This  study  also  showed  that  dental 
school  class  standing,  by  itself,  is  of  little  practical  significance  as  a  tool  for  selecting  AEGD 
participants.  In  addition,  the  results  of  this  study  do  not  support  the  concerns  voiced  by  program 
mentors  that  AEGD  performance  has  diminished  over  time. 
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Introduction  and  overview 


Purpose 

This  report  presents  the  results  of  a  1992  study  initiated  to 
examine  factors  associated  with  success  in  the  one  year  Advanced 
Education  in  General  Dentistry  (AEGD)  Program  sponsored  by  the  U.S. 
Army  Dental  Corps  and  to  determine  how  well  new  accessions  with 
lower  dental  school  class  standings  will  be  likely  to  succeed  in 
the  AEGD  residency.  This  report  provides  the  Army  Dental  Care 
System  leadership  with  information  and  a  framework  for  evaluating 
the  existing  AEGD  selection  process. 

Background 

Trends  in  the  practice  of  general  dentistry  dictate  the  need  for 
proficiency  in  all  the  clinical  disciplines  (1)  .  A  one-year 
postdoctoral  AEGD  program  can  provide  recent  graduates  with  the 
opportunity  to  increase  their  knowledge  in  all  disciplines  and  to 
develop  speed,  confidence,  and  advanced  techniques  under  faculty 
supervision  (2)  .  Accredited  AEGD  programs  have  expanded 
significantly;  from  1981  to  1986  their  numbers  increased  from  4  to 
31.  Forty-two  percent  of  these  programs  were  sponsored  by  the 
federal  dental  services  (3). 

The  American  Association  of  Dental  Schools  (AADS)  Task  Force 
on  Advanced  Education  and  the  American  Dental  Association  (ADA) 
Special  Committee  on  the  Future  of  Dentistry  in  the  1980s  have 
recommended  support  for  postdoctoral  education  programs  in  general 
dentistry  (4) .  In  1987,  the  AADS  Council  of  Hospitals  sponsored  a 
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symposium  to  discuss  concepts  and  present  practical  approaches 
related  to  implementing  required  postdoctoral  education  programs  in 
general  dentistry  for  all  dental  school  graduates  (4) .  At  the  1992 
annual  meeting  of  the  AADS,  Resolution  11  called  for  the  continuing 
development  of  postdoctoral  programs  in  general  dentistry  so  that 
by  the  year  2000,  all  graduates  of  U.S.  dental  schools  will  be 
required  to  enroll  in  one  year  of  postdoctoral  training  (5) . 
However,  until  a  consensus  of  support  is  reached  among  appropriate 
licensing  bodies,  policy  makers,  educators,  practitioners,  and 
sponsors  of  programs,  the  debate  surrounding  this  resolution  for  a 
postdoctoral  education  program  ir  general  dentistry  is  likely  to 
continue. 

AEGD  and  the  US  Army  Dental  Corps 

Such  innovations  that  are  evolving  at  the  national  level,  will 
have  a  profound  impact  upon  the  Army's  postdoctoral  programs.  If 
the  proposed  resolution  requiring  postdoctoral  AEGD  is  adopted,  the 
Army  will  be  faced  with  choosing  to  continue  its  existing  programs 
or  to  acquire  civilian-trained  new  dentists.  This  complex  choice 
will  have  to  be  evaluated  in  terms  of  current  budgetary 
constraints,  the  importance  of  the  programs  to  the  military,  and 
consequences  of  the  choice.  For  the  purposes  of  this  paper,  it  is 
assumed  that  a  continuation  of  existing  AEGD  programs  will  be  the 
Army's  choice. 

Given  this  choice,  what  selection  criteria  will  the  Army  employ 
to  ensure  that  the  best  possible  applicants  are  selected  for  its 
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Army's  AEGD  program?  Will  the  current  selection  process  for  AEGD 
remain  intact  or  will  it  be  modified?  These  are  questions  for  the 
Army  Dental  Care  System  leaders.*ip  and  will  require  further  study. 

Traditionally,  the  Army's  selection  process  for  AEGD  has 
involved  a  review  of  an  applicant's  qualifications  by  a  selection 
board  composed  of  senior  dental  officers.  Information  available  to 
the  board  usually  consists  of  background  information  such  as  age, 
gender,  dental  school  attended,  class  rank  in  dental  school, 
transcripts,  a  history  of  prior  military  service,  and  letters  of 
recommendation.  The  board  then  ranks  applicants  and  offers  AEGD 
positions  based  on  that  ranking. 

In  recent  years,  because  of  fewer  applicants,  the  Army  has  had 
to  select  a  number  of  individuals  with  dental  school  standing  below 
the  class  mid-point.  In  addition,  program  mentors  have  expressed 
concern  that  recent  Army  AEGD  participants  have  not  been  performing 
as  well  as  residents  did  in  the  past.  Presumably  this  is  because 
of  the  new  residents'  poorer  academic  performance  in  dental  school. 
These  concerns  were  clearly  demonstrated  in  the  selection  process 
for  the  1991  and  1992  AEGD  classes;  only  32  and  40  applicants, 
respectively,  were  selected,  although  there  were  42  positions 
available  in  1991  and  48  positions  were  available  in  1992.^  If 

the  proposed  resolution  for  required  postdoctoral  AEGD  is  adopted, 
it  is  likely  that  the  additional  competition  for  applicants  will 

^Personal  communication  from  Colonel  Jon  Rampton,  Chief, 
Graduate  Dental  Education  for  the  US  Army  Dental  Corps,  August 
1992. 
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require  the  Army  to  accept  individuals  with  even  poorer  acadenic 
credentials. 

AEOD  Selection  Criteria  Developaent 

Selection  of  personnel  for  an  AEGD  program  is  a  complex  process. 
One  of  the  problems  facing  selection  boards  is  how  to  choose  from 
the  applicant  pool  those  candidates  most  likely  to  successfully 
complete  the  AEGD  program  of  study.  The  primary  question  becomes, 
"Using  the  available  application  information,  how  does  a  selection 
board  identify  those  applicants  most  likely  to  succeed?” 

Ideally,  the  selection  board  should  begin  with  clearly 
formulated  criterion-related  strategies  for  evaluating  the 
applicant's  potential  for  success  in  the  AEGD  program  (6). 
However,  this  is  usually  not  the  case.  The  process  more  often 
involves  the  selection  board's  reviewing  available  biographical 
information,  prior  accomplishments,  and  demonstrated  academic 
performance  to  ultimately  form  an  inference  about  an  applicant's 
probable  success  in  the  program.  This  method  of  selection  is  based 
on  the  behavioral-consistency  model  premise  that  "the  best 
indicator  of  future  performance  is  past  performance” (7) .  To  date, 
this  has  not  been  proven  with  regard  to  the  selection  of  applicants 
for  the  Army's  AEGD  program. 

Many  techniques  have  been  employed  for  assessment  of  personnel. 
One  of  the  most  widely  used  devices  has  been  the  biographical 
inventory,  whether  as  a  personal  history  inventory,  an  application 
blank,  or  even  the  schedule  for  a  selection  interview  (8)  . 
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Biographical  inventories  have  consistently  been  shown  to  provide 
valid  predictors  of  criteria  relevant  to  organizational  success 
(8) .  Traditionally,  these  criteria  have  been  tied  to  job 
performance  or  other  occupationally  specific  measures;  and  they 
have  involved  such  diverse  constructs  as  sales  success  (9)  ,  career 
paths  (10),  and  general  career  success  (8).  Hough  applied  the 
principle  of  behavioral  consistency  and  aspects  of  the  biographical 
inventory  to  develop  and  validate  the  ” Accomplishment  Record" 
method  for  selection,  promotion,  and  performance  appraisal  of  legal 
professionals  (11).  Campbell  used  a  combination  of  hands-on  job 
sample  tests,  multiple-choice  knowledge  tests,  peer  and  supervisor 
ratings,  and  existing  file  data  to  develop  criteria  to  validate 
both  experimental  and  existing  predictors  of  job  performance  among 
entry-level  enlisted  positions  in  the  U.S.  Army  (12). 

The  application  of  assessment  techniques  in  predicting  the 
success  of  personnel  has  not  been  ignored  by  the  field  of  dental 
education.  Staat  and  Yancey  used  a  combination  of  preprofessional 
academic  records  and  selected  personal  characteristics  to  develop 
the  "Admission  Index"  in  an  attempt  to  improve  the  predictability 
of  success  in  the  admission  process  for  dental  schools  (13). 
Potter  and  McDonald  used  structural  models  to  examine  the  direct 
impact  of  the  quality  cf  dental  school  education  on  future 
professional  performance  (14).  Scheetz  used  discriminant  analysis 
to  determine  if  admissions  data  available  to  admissions  committees 
was  useful  in  distinguishing  between  potential  graduates  and 
nongraduates  of  dental  schools  (15).  Westerman  assessed 
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personality  preferences  as  predictors  of  performance  for  first-year 
dental  students  (16).  Simon  and  Chambers  administered  a  battery  of 
22  aptitude  tests  to  a  national  sample  of  dentists  in  an  attempt  to 
predict  success  in  dental  training  and  in  dental  career 
performance  (17). 

As  thorough  as  the  research  has  been  in  exploring  measures 
associated  with  occupational  success,  very  little  wor)c  involving 
measures  associated  with  success  in  postgraduate  dental  programs 
has  been  reported.  And,  to  date,  no  published  research  has 
reported  on  measures  of  success  for  those  in  postgraduate  dental 
programs  or  predictors  of  such  success. 

Recent  studies  have  utilized  outcomes  assessment  in  order  to 
evaluate  postgraduate  dental  programs.  Wolf  measured  the 
satisfaction  of  residents  who  had  completed  general  practice 
residency  programs  (18) .  Handelman  and  Demby  and  Burakoff  measured 
the  effect  of  general  practice  residency  training  on  residents' 
subsequent  practice  of  dentistry  (19,20), 

One  of  the  biggest  handicaps  associated  with  predicting  success 
is  that  success  is  a  difficult  concept  to  conceptualize  and  to 
define  operationally.  Assuming  that  the  problems  involved  in 
defining  and  measuring  success  can  be  overcome,  identifying  those 
biographical  and  academic  factors  associated  with  success  in  a 
given  program  would  provide  very  useful  guidelines  for  the 
selection  of  applicants  for  AEGD, 

The  present  study  had  two  goals:  1)  to  identify  those  background 
factors  associated  with  a  person's  success  in  the  AEGD  program,  and 
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2)  to  examine  the  likelihood  that  new  accessions  with  lower  dental 
school  class  standings  will  succeed  in  the  Army's  AEGD  program. 
This  study  accomplishes  this  by  developing  a  success  criterion, 
i.e.,  a  measure  of  the  major  dimensions  of  performance  during  AEGD 
residency,  against  which  factors  identified  as  predictors  of 
success  in  the  AEGD  program  can  be  validated. 
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Methods 


Determination  of  the  Dimensions  of  AE6D  Resident  Performance 

Each  U.S.  Army  AEGD  participant  receive  a  formal  academic 
evaluation  report,  the  US  Army  Officer  Evaluation  Report  (OER)  DA 
Form  67-8,  at  the  end  of  their  program.  This  provides  the 
opportunity  for  the  AEGD  program  mentor  (rater)  and  another  senior 
dental  officer  (senior  rater)  involved  with  the  program  to 
critically  evaluate  the  AEGD  participant  in  the  following  respects; 

The  Rater  •  Professionalism 

•  Ability  to  meet  Program  Requirements 

•  Comment  on  Specific  Aspects  of  Performance 

•  Potential  for  Promotion  to  Next  Highest  Ran)c 

•  Comment  on  overall  Potential 

Senior  Rater  •  Comments  on  Overall  Potential 

This  report  then  becomes  part  of  the  participant's  Official 
Military  Personnel  File  which  is  maintained  on  centrally  located 
microfiche  and  is  utilized  by  the  Army  Dental  Corps  in  determining 
assignments,  promotions,  and  consideration  for  further  professional 
training  programs. 

In  an  attempt  to  identify  the  major  dimensions  of  performance  in 
the  AEGD  program,  25  anonymous  copies  of  academic  evaluation 
reports,  representing  3  different  AEGD  class  cohorts,  were  obtained 
from  the  Airmy's  Office  of  the  Dental  Corps  Branch.  Each  academic 
report  was  independently  reviewed  and  analyzed  for  content  by  three 
experienced  dental  officers.  Each  reviewer  subjectively  formed 
preliminary  themes  of  overall  accomplishments  within  the  content  of 
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the  reports  (21) .  No  strict  rules  were  imposed  on  the  three  dental 
officers  in  the  selection  of  these  major  themes. 

The  next  phase  was  a  Delphi-type  negotiation  among  the  three 
reviewers  wherein  they  merged  their  choices  into  a  single  list  of 
the  major  dimensions  of  performance  in  AEGD.  Information  about  the 
choices  and  the  rationale  for  them  was  discussed  among  the  three 
reviewers.  As  a  result  of  this  group  discussion,  five  areas  were 
agreed  upon  as  represente tive  of  the  dimensions  of  performance 
during  the  AEGD  residency.  They  were  designated  (a)  Institutional 
Potential,  (b)  Leadership  Potential,  (c)  Dental-Professional 
Potential,  (d)  Academic  Performance,  and  (e)  Overall  Residency 
Performance,  Criteria  were  written  to  define  each  of  these  five 
dimensions. 

Development  of  the  AEGD  Success  Index 

The  procedures  just  described  yielded  the  major  dimensions  of 
performance  by  AEGD  residents  and  provided  the  structure  for 
developing  the  Success  Index  for  AEGD. 

The  Army's  Office  of  the  Dental  Corps  Branch  provided  Academic 
Evaluation  Reports  (AERs) ,  along  with  background  biographical 
information,  on  a  representative  sample  of  recent  AEGD 
graduates  (n=90)  .  Appendix  A  contains  a  copy  of  the  AER  and  the 
Background  Biographical  Information  Data  Collection  Sheet. 
Confidentiality  of  reports  and  records  was  maintained  throughout 
the  study.  AEGD  cohorts  from  1984  (n=33) ,  1986  (n=24) ,  and  1987 
(n=33)  were  selected  in  order  to  examine  applicant  selection  and 
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program  trends,  as  well  as  to  create  the  opportunity  to  evaluate 
how  measures  of  success  in  AEGD  translate  into  career  success. 

The  90  AERs  were  reviewed  by  two  experienced  senior  dental 
officers.  Using  a  modification  of  the  method  developed  by  Hough 
(1984)  ,  each  reviewer  was  asked  to  rate  the  reports  in  each  of  the 
five  AEGD  program  dimensions  (11)  .  In  each  dimension,  the  AEGD 
participant  was  rated  on  a  0  to  4  ordinal  scale  with  "0"  assigned 
to  the  lowest  and  'M”  representing  the  highest  level  of 
achievement.  Guidelines  specifying  elements  to  look  for  or 
principles  to  use  when  rating  the  level  of  achievement  within  the 
five  program  dimensions  were  given  to  the  two  reviewers  (see 
Appendix  B  for  the  Evaluation  Form  and  guidelines  used) . 

A  rating  score  on  each  of  the  five  program  dimensions  and  an 
overall  score,  representing  the  sum  of  the  five  individual  scores, 
were  tabulated  by  each  reviewer  for  each  AER.  The  reliabilities  of 
the  reviewers'  ratings  were  estimated  using  the  kappa  statistic  and 
percent  absolute  agreement  (22) .  As  can  be  seen  in  Table  l,  the 
dimensions  most  reliably  rated  by  the  reviewers  were  dental 
professional  potential  and  overall  residency  performance  (.80)  ,  the 
dimension  least  reliably  rated  was  institutional  potential  (.55), 
and  the  median  reliability  was  .72.  Fleiss  (22)  and  Landis  and 
Koch  (23)  have  suggested  that  for  interpretation  of  kappa,  0.40  and 
below  represents  poor  agreement  beyond  chance,  0.40-0.75  represents 
fair  to  good  agreement,  and  0.75  and  above  represents  excellent 
agreement.  This  analysis  suggested  that  the  reviewers  had 
evaluated  the  AER  with  sufficient  reliability  to  warrant  the  use  of 
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the  five  dimensions  in  the  development  of  an  index  of  success  in 
AEGD.  The  t-test  statistic  was  used  to  compare  mean  scores  by 
reviewer,  across  all  five  dimensions  for  each  of  the  three  year- 
group  cohorts.  No  statistically  significant  differences  in  scoring 
were  detected  at  the  p  <  .05  level. 

All  scores,  totals  (called  overall  success  scores)  and  those  for 
each  of  the  five  dimensions,  were  pooled  for  the  two  reviewers. 
Mean  values  were  calculated  and  then  used  as  criteria  to  index 
success  in  AEGD.  The  success  criteria  measures  and  descriptions, 
mean  scores,  and  standard  deviations  are  presented  in  Table  2. 
Because  it  was  felt  that  the  five  dimensions  of  success  in  the  AEGD 
program  represent  somewhat  different  but  interrelated  constructs, 
principal  component  analysis  was  performed  to  simplify  the 
description  of  the  dimensions.  The  final  one-factor  solution 
(named  Program  Success)  was  selected  on  the  basis  of  the  proportion 
of  variance  (0.83)  explained  by  the  first  principal  component  (24). 
Table  3  presents  the  principal  component  patterns  for  the  five 
dimensions  of  success  in  AEGD.  As  can  be  seen  in  Table  3,  no 
single  coefficient  distinguished  itself  as  highly  correlated  with 
the  first  principal  component.  This  was  interpreted  as  meaning 
that  the  first  principal  component  was  reasonably  representative  of 
the  five,  original  dimensions,  and,  thus,  was  used  to  express 
overall  success  in  an  AEGD  program. 
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overview  of  the  AE6D  Success  Index  Development 

After  the  important  dimensions  of  the  AEGD  program  were 
established,  rating  principles  and  scales  were  developed  for 
reviewers  to  use  when  evaluating  an  AEGD  participant's  AER.  This 
method  resulted  in  (a)  definitions  of  the  important  dimensions  of 
the  AEGD  program;  (b)  summary  principles  that  highlight  key 
characteristics  to  look  for  when  determining  the  level  of 
achievement  demonstrated  within  the  content  of  the  evaluation;  and 
(c)  numerical  equivalents  that  allow  the  accomplishments  within 
each  dimension  to  be  translated  into  quantitative  indices  of 
success . 

Predictors 

Although  it  is  acknowledged  that  many  factor:  potentially 
influence  success  in  the  AEGD  program,  only  those  biographical 
factors  available  to  selection  panels  were  examined. 
Characteristics  of  the  factors  examined  appear  in  Table  4 . 

Results 

Factors  Associated  with  the  AEGD  Success  Index 

Simple  correlation  coefficients  between  the  background  factors 
and  the  AEGD  success  index  scores  are  presented  in  Table  5.  A 
Regular  Army  appointment  {r=.25),  overall  class  standing  in  dental 
school  {r=.26),  and  prior  military  education  (r=.23,  either  in  the 
Reserve  Officer  Training  Corps  or  at  the  U.S.  Military  Academy), 
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were  all  significantly  and  positively  correlated  with  the  Program 
Success  Index  score. 

Multiple  stepwise  regression  analysis,  using  the  Statistical 
Analysis  System",  was  performed  to  relate  all  of  the  background 
factors  described  in  Table  4  to  the  Program  Success  Index  scores. 
The  background  factors  identified  were  prior  military  education, 
dental  school  class  standing,  graduation  age,  a  Regular  Army 
appointment,  and  the  interaction  term  of  prior  military  education 
with  graduation  age.  The  results  of  the  regression  analysis  are 
presented  in  Table  6.  The  model  relating  factors  from  the 
backgrounds  of  AEGD  participants  to  the  Program  Success  Index  score 
was  statistically  significant  (F  value  =  4.42,  p  <  .0013),  and  the 
linear  combination  of  the  factors  accounted  for  20.83  percent  of 
the  variation  in  the  Program  Success  Index  score.  This  coefficient 
of  multiple  determination,  R^,  is  comparable  to  levels  reported  in 
other  occupationally  specific  models  (8).  All  of  the  background 
factors  identified,  with  the  exception  of  a  Regular  Army 
appointment  (p  <.0598),  were  significant  at  the  p  <  .05  level. 

In  order  to  evaluate  whether  the  Overall  Success  Score  (the  sum 
of  the  five  dimensional  scores)  would  yield  a  higher  coefficient  of 
determination  than  would  the  principal  component  score,  a  multiple 
regression  model  using  the  same  background  factors  was  executed. 
The  R^  value  was  .2015  (model;  F  value  =  4.241,  p  <  .0017) 
indicating  that  the  simple  addition  of  the  dimensional  rating 
scores  gives  essentially  the  same  results  as  using  the  first 
principal  component  (Program  Success  Index  score)  for  this  one- 
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regression  comparison.  Table  7  presents  examples  of  how  changes  in 
the  background  factors  will  alter  the  Overall  Success  Score. 

AE6D  Success  Based  on  Class  Standing 

One  of  the  primary  goals  of  the  study  was  to  examine  the 
likelihood  that  new  accessions  with  lower  dental  school  class 
standings  will  succeed  in  the  Army's  AEGD  program.  The  model 
compared  AEGD  participants  who  had  graduated  in  the  top  half  of 
their  dental  school  classes  to  those  who  had  graduated  in  the 
bottom  half.  Estimates  for  class  standing  in  Table  6  can  be 
interpreted  as  follows:  compared  to  those  graduating  in  the  bottom 
half  of  their  class,  AEGD  participants  who  graduated  in  the  top 
half  would  be  expected  to  have  nearly  a  two-point  increase  in  their 
estimated  Program  Success  Index  score,  holding  all  other  variables 
constant.  A  similar  model,  comparing  participants  in  the  top  one- 
third  of  their  class  with  those  in  the  bottom  half  of  their  class, 
failed  to  establish  class  standing  as  a  significant  factor  at  the 
p  <  .05  level,  even  though  the  overall  regression  model  was 
significant  (model:  F  value  =  2.945,  p  <  .0235,  =  .2691). 

To  further  examine  the  effect  that  dental  school  class  standing 
had  on  success  in  the  AEGD  program,  the  t-test  statistic  was  used 
to  compare  the  mean  values  of  the  factors  and  the  Program  Success 
Index  scores  between  AEGD  participants  in  the  top  half  of  their 
graduating  dental  class  (n=71)  and  those  in  the  bottom  half  (n=19) . 
The  only  significant  difference  detected  between  the  two  groups  was 
in  grade  point  average  from  dental  school  (Bottom  half:  GPA=2.83, 
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St. dev. =.26,  Top  half;  GPA=3.08,  st.dev.=.27,  p  <  .0004). 

Comparisons  of  Program  Success  among  Year  Groups 

Stratification  by  year  group  cohort  permitted  the  validation  of 
the  concerns  voiced  by  mentors  in  the  AEGD  program  that  the 
performance  of  recent  AEGD  participants  was  lower  than  in  the  past. 
Differences  in  mean  Program  Success  scores  were  examined  using 
analysis  of  variance  (ANOVA)  and  t-tests  for  multiple  comparisons 
of  mean  values.  The  ANOVA  revealed  that  the  mean  Program  Success 
scores  for  the  three  year  groups  were  not  all  equal  (F  =  3.49, 
p  =.0348).  Significant  differences  were  detected  between  the  1986 
and  1987  year  groups  (t=  2.45,  p  <  .0173).  However,  the  variances 
of  these  two  groups  were  equal  (F  =1.14,  p  <  .7486).  No 

significant  differences  were  detected  between  the  1984  and  1986 
cohorts,  or  between  the  1984  and  1987  cohorts.  No  clear  pattern  of 
decreased  performance  was  evident  from  the  analysis  of  these  three 
year  groups. 

Predicting  Success  among  Program  Participants 

A  probit  logistic  regression  analysis  was  used  to  assess  the 
usefulness  of  the  predictive  factors  in  classifying  individuals  as 
either  above  average  or  below  average  in  terms  of  success  in  the 
AEGD  program  (25) .  The  two  levels  of  success  in  the  AEGD  program 
were  arbitrarily  defined  as  above  or  below  the  mean  Overall  Success 
score.  The  logistic  regression  procedure  correctly  classified  67.8 
percent  of  the  AEGD  graduates  as  either  successful  or  as  less  than 
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successful  (Table  8) .  Thus,  it  is  evident  that  this  logistic  model 
(X^  =17.96,  df=5,  p  ==.003),  utilizing  the  identified  biographical 
factors,  provided  a  slightly  better  than  chance  probability  of 
correctly  classifying  participants  as  above  or  below  average  in 
terms  of  success  in  the  AEGD  program. 

Odds  ratios  were  computed  comparing  the  odds  of  being  classified 
as  an  above  average  performer  verses  a  below  average  performer  in 
terms  of  program  success  based  on  class  standing  (25)  .  For  this 
model,  the  odds  that  someone  who  graduated  in  the  top  half  of  a 
class  would  be  classified  as  being  an  above  average  performer  were 
1.55  times  greater  (95%  Confidence  Interval:  1.34,  1.76)  than  for 
someone  who  graduated  in  the  bottom  half  of  a  class. 

AEGD  Success  and  Career  Success 

The  career  progressions  of  the  three  year  group  cohorts  were 
followed  from  the  time  of  completion  of  the  AEGD  program  until 
1991.  The  Dental  Corps  Career  Activities  Office  supplied  data  from 
Officer  Efficiency  Reports  (OERs) ,  promotion  to  the  rank  of  major, 
and  selection  for  dental  specialty  training  for  each  of  the  AEGD 
graduates.  To  further  examine  the  effect  that  dental  school  class 
rank  and  success  in  the  AEGD  program  had  on  career  success  in  the 
Dental  Corps,  z-approximation  tests  were  performed  to  compare  the 
proportion  of  those  graduating  in  the  top  half  of  their  class  with 
those  graduating  in  the  bottom  half  of  their  class.  The  following 
criteria  were  used:  (a)  the  proportion  receiving  top-block  ratings 
on  their  OERs;  (b)  the  proportion  promoted  to  the  rank  of  major. 
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ahead  of,  or  with,  their  peers;  and  (c)  the  proportion  selected 
with,  or  ahead  of,  their  peers  for  advanced  dental  specialty 
training. 

Comparing  the  career  success  of  those  graduating  in  the  top  half 
of  their  dental  school  class  with  those  graduating  in  the  bottom 
half,  no  significant  differences  were  detected  for  any  of  the  three 
criteria  outlined  above.  When  the  AEGD  graduates  were  stratified 
according  to  whether  they  were  above  or  below  the  mean  Overall 
Success  score,  no  significant  differences  were  detected  for  the 
proportion  of  AEGD  graduates  who  received  top-block  OER  ratings  or 
for  the  proportion  who  were  selected  ahead  of,  or  with,  their  peers 
for  promotion  to  major.  Significant  differences  were  revealed  for 
the  proportion  of  those  selected  ahead  of  their  peers  for  advanced 
specialty  training  (.205  *  below  the  mean  Overall  Success  score, 
.451  =  above,  p=.015);  and  significant  differences  were  found  in 
regard  to  overall  selection  for  specialty  training  between  the 
proportion  of  those  above  the  mean  Overall  Success  score  and  those 
below  (above=,608  vs.  below=.231,  p=.0001). 

Siuunary  of  Results 

Consensus  among  the  reviewers  determined  the  major  dimensions  of 
performance  in  the  AEGD  program.  Those  dimensions  were  used  to 
create  a  one  dimensional  Program  Success  Index.  Background  factors 
readily  available  to  selection  panels  were  then  related  to  the 
Program  Success  Index  using  an  ordinary  least  squares  multiple 
regression  model.  Prior  military  education,  dental  school  class 
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standing,  graduation  age,  a  Regular  Army  appointment,  and  an 
interaction  term  (combining  prior  military  education  with  age  at 
graduation)  emerged  as  significant  predictors  of  success  in  the 
AEGD  program,  and  these  factors  accounted  for  approximately  21 
percent  of  the  variation  in  Program  Success  Index  scores. 

In  a  logistic  regression  model,  the  identified  predictors 
correctly  classified  67.8  percent  of  the  AEGD  participants  in  terms 
of  success  in  the  AEGD  program.  And,  in  this  model,  AEGD 
participants  from  the  top  half  of  their  dental  school  class  were 
1.55  times  as  likely  as  those  in  the  lower  half  to  be  classified  as 
above  average  in  terms  of  program  success. 

Class  standing  by  itself  did  not  affect  the  Program  Success 
Index  score  or  subseguent  career  success  as  measured  by  OERs, 
promotion  to  the  rank  of  major,  or  selection  for  advanced  dental 
specialty  education. 
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Discussion 


The  contribution  of  this  study  lies  primarily  in  two  areas:  it 
operationally  defines  success  in  an  AEGD  program,  and  it  identifies 
background,  application  data  that  are  significantly  associated  with 
success  in  an  AEGD  program. 

Given  the  relatively  little  work  done  in  measuring  postgraduate 
dental  program  success,  the  issue  of  the  validity  of  the  Program 
Success  Index  score  needs  further  elaboration.  The  use  of  three 
"experts"  to  determine  the  dimensions  of  accomplishments  inferred 
from  academic  reports  from  AEGD  seems  consistent  with  similar 
methods  reported  in  the  literature.  Staat  and  Yancey  used  mutual 
agreement  from  university  administrators  to  develop  a  formula  for 
the  "Admission  Index"  in  the  dental  school  admissions  process  (13). 
Hough  used  three  "expert"  attorneys  to  rate  the  level  of 
achievement  demonstrated  by  law  professionals  on  an  "Accomplishment 
Record  Inventory"  (11) .  Peterson  used  pooled  expert  judgments  of 
personnel  psychologists  to  enumerate  an  appropriate  list  of 
criterion  content  categories  for  making  selection  and 
classification  decisions  for  entry-level  military  personnel  (26) . 

It  is  interesting  that  the  five  dimensions  of  achievement  agreed 
upon  by  the  experts  used  in  this  study  fit  reasonably  well  within 
the  four  classes  of  outcome  assessment  measures 
(Knowledge  Outcomes,  Skill  Outcomes,  Values/Beliefs,  and 
Relationship/Behavioral  Measures)  of  student  academic  achievement 
outlined  by  the  American  Dental  Association  Commission  on  Dental 
Accreditation  (27) .  The  good  to  excellent  reliabilities  (Table  1) 
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of  the  expert  reviewers  in  this  study  lend  additional  support  to 
the  validity  of  the  success  dimensions. 

Another  area  that  deserves  discussion  relates  to  the  predictor 
factors  identified.  When  the  background  application  factors 
traditionally  used  in  the  selection  process  were  examined 
individually,  only  three  of  them  (a  Regular  Army  appointment,  prior 
military  education,  and  class  standing)  were  even  weakly  correlated 
with  the  Index  of  Program  Success  score  (Table  5) .  The  overall 
lack  of  correlation  between  these  background  factors  and  success  in 
the  AEGD  program  is  a  pattern  consistent  with  the  modest 
correlations  others  have  reported,  especially  with  regard  to  the 
lack  of  correlation  between  background  admission  factors  and  dental 
school  performance  (13,15,28).  Similar  results  have  been  reported 
for  other  professions.  Hough  reported  essentially  no  correlation 
between  traditional  background  factors  such  as  honors,  grades,  Law 
School  Aptitude  Test  scores,  and  the  bar  examination  results  with 
the  "Accomplishment  Record"  score  developed  for  selecting  and 
promoting  legal  professionals  (11) . 

A  possible  explanation  for  this  lack  of  correlation  is  that  the 
AEGD  participants  used  in  this  study  constitute  a  relatively 
homogeneous  population  relative  to  the  precision  of  the  Program 
Success  Index.  Graduation  from  dental  school  ensures  that  each 
AEGD  participant  has  the  basic  body  of  knowledge  and  the  clinical 
skills  necessary  for  minimal  success  in  the  clinical  curriculum  of 
AEGD.  And,  as  Staat  and  Yancey  observed,  predictors  have  little 
value  in  a  homogeneous  population  (13).  On  the  other  hand,  as  the 
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participants  progress  through  the  program,  subsequent  performance 
may  be  influenced  by  motivation  and  maturity  as  Helmreich 
reported  (29).  The  factors  significantly  correlated  with  the 
Program  Success  Index  (a  Regular  Army  appointment,  prior  military 
education,  and  class  standing)  discriminate  between  an  otherwise 
homogeneous  population  and  serve  as  indicators  of  motivation  for 
success  in  the  Army  and  in  the  AEGD  program. 

The  results  of  the  multiple  regression  analysis  held  few 
surprises  given  the  sample  specificity  of  this  statistical 
technique.  With  the  exception  of  the  interaction  term  representing 
those  older  individuals  who  had  prior  military  education,  all  of 
the  identified  factors  had  positive  influences  on  the  Program 
Success  Index  score.  Initially  it  was  thought  that  older 
participants  with  prior  military  education  would  exhibit  the 
maturity  of  age  and  familiarity  with  the  military  system  necessary 
to  succeed  in  the  AEGD  program.  It  is  possible  that  these 
individuals  were  held  to  higher  perfoirmance  expectations  than  those 
less  familar  with  the  Ajrmy  and  that  this  effected  their  overall 
Success  Index  scores.  However,  due  to  the  small  number  of  AEGD 
participants  in  this  category  (n=7),  interpretation  of  this 
factor's  influence  must  be  approached  cautiously. 

Of  the  background  factors  detailed  in  Table  6,  class  standing 
accounted  for  the  greatest  amount  of  explained  variance  in  the 
regression  model  (33.6  percent).  A  Regular  Army  appointment  and 
graduation  age  contrii>uted  24  and  and  19  percent,  respectively.  As 
was  pointed  out  above,  the  amount  of  variance  in  the  Program 
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Success  Index  scores,  explained  by  the  linear  combination  of  the 
identified  factors  (R^  =  .2083),  is  comparable  to  the  levels 
reported  in  other  occupationally  specific  models  (8) .  These 
results  confirm  that  factors  other  than  those  available  to 
selection  committees  are  responsible  for  much  of  the  variation  seen 
in  the  patterns  of  success  among  AEGD  participants. 

The  usefulness  of  the  predictive  factors  in  classifying 
individuals  as  either  above  average  or  below  average  in  terms  of 
their  success  in  the  AEGD  program  met  with  mixed  success.  Although 
the  probit  logistic  model  successfully  classified  67.8  percent  of 
the  AEGD  participants,  this  result  was  interpreted  as  providing 
only  a  slightly  better  than  chance  probability.  These  results  were 
comparable  to  those  of  Scheetz,  who  classified  individuals  as  to 
the  likelihood  of  graduation  from  dental  school  using  admissions 
data  readily  available  to  admissions  committees  (15) .  Although  not 
directly  comparable,  the  results  of  this  study  and  those  reported 
by  Scheetz  have  similar  explanations,  namely,  that  factors  not 
readily  available  to  selection  committees  contribute  to  success 
within  a  program. 

The  results  of  this  study  do  not  support  the  concerns  expressed 
by  the  program  mentors,  i.e.,  that  current  or  recent  AEGD 
participants  have  not  performed  as  well  as  participants  have  in  the 
past.  Even  though  the  ANOVA  analysis  revealed  that  mean  Program 
Success  scores  for  the  three  year  groups  were  not  all  equal,  no 
significant  differences  were  detected  between  the  1984  and  1986 
cohorts,  or  between  the  1984  and  1987  cohorts.  The  only  detectable 
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difference  was  between  the  1986  and  1987  year  groups  and,  even 
then,  the  variance  of  mean  Program  Success  scores  between  these  two 
groups  was  statistically  the  same.  Further,  no  significant 
differences  were  detected  in  mean  class  standing  when  comparisons 
were  made  between  the  three-year  group  cohorts.  Perhaps  the 
examination  of  a  wider  range  of  year  groups  would  reveal  a  pattern 
of  decreased  performance  not  revealed  in  this  study. 

Thus,  for  this  sample  of  AEGD  participants,  dental  school  class 
standing  was  not  sufficiently  strong  as  a  single  predictor  of 
success  in  the  AEGD  program,  nor  were  there  significant  differences 
in  mean  class  standing  between  the  three  cohorts.  However,  it  must 
be  pointed  out  that  when  class  standing  was  considered  in 
combination  with  other  background  factors,  it  did  make  a 
significant  contribution  to  distinguishing  between  different  levels 
of  success  within  the  program. 

The  results  of  this  study  are  not  without  possible  bias.  There 
are  many  problems  inherent  in  defining  and  measuring  program 
success.  We  have  chosen  to  define  success  in  AEGD  from  only  one 
perspective;  that  of  the  mentors,  and  we  have  only  measured  that 
success  from  a  content  analysis  of  the  AER.  The  development  of  the 
Program  Success  Index  hinges  upon  the  assumption  that  the  AER  is  an 
accurate  reflection  of  resident's  performance.  It  is  possible  that 
these  evaluations  reflect  the  ability  of  the  program  mentor  to 
accurately  describe  the  resident's  performance  more  than  they 
reflect  that  performance  itself.  The  perception  of  a  resident's 
performance,  as  detailed  in  the  AER  and  when  viewed  by  selection 
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boards  or  even  the  expert  reviewers  in  this  study,  can,  thus,  be 
biased  by  a  mentor's  skill  and  experience  in  writing  AERs.  As  Guion 
and  Gibson  reported,  performance  ratings  can  also  be  biased  by  the 
personal  relationship  between  the  resident  and  the  mentor  (6)  . 
Even  though  these  influences  may  effect  the  assessment  of 
performance  by  reviewers  of  an  AER,  it  is  felt  that  the  dimensions 
of  success  in  AEGD  uncovered  in  this  study,  would  not  be  altered  by 
this  potential  bias. 

The  issue  of  institutional  variation  must  also  be  considered  as 
a  potential  influence  on  the  performance  evaluation.  Program 
mentors  use  program  objectives  of  the  ADA  Commission  on 
Accreditation  as  guidelines,  but  AEGD  program  objectives  can  still 
vary  among  training  institutions  due  to  differences  in  facilities 
and  staffing,  the  experience  and  training  of  the  mentor,  and  what 
has  been  done  previously  in  the  program.  Success  at  any  one 
institution  then  becomes  a  matter  of  how  well  a  resident  meets  the 
objectives  and  expectations  of  that  program,  as  viewed  by  the 
program  director.  Because  program  directors  have  no  standardized 
objectives,  they  (currently  there  are  six  programs)  can  be  very 
independent  and  the  "institutional  personality"  that  develops  can 
cause  success  to  be  viewed  very  differently  from  program  to 
program.  If  one  assumes  that  residents  and  mentors  (and  their 
aptitudes)  are  nondif ferentially  assigned  to  the  various  training 
sites,  and  that  training  programs  operate  under  the  same  general 
program  objectives,  then  one  could  expect  to  see  little  or  no 
difference  in  resident  performance  from  program  to  program  and  from 
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year  to  year.  The  results  of  this  study  showed  this  to  be  true. 
Therefore,  it  can  be  concluded  for  this  sample  population,  that, 
while  the  training  programs  may  be  very  independent,  "institutional 
personality”  did  not  appear  to  influence  success  as  viewed  by  the 
mentors . 

Given  the  sample  specific  predictors  identified  in  this  study, 
we  are  reluctant  to  presume  that  the  same  factors  would  emerge  with 
another  non-military  sample.  The  question  of  the  universality  of 
these  factors  notwithstanding,  the  primary  value  of  the  novel 
approach  used  in  this  study  is  in  helping  to  further  clarify  the 
nature  of  success  in  AEGD  through  the  development  of  the  Program 
Success  Index.  It  is  suggested  that  the  five  dimensions  of  success 
uncovered  in  this  study  can  serve  as  a  framework  for  establishing 
evaluative  criteria.  The  results  suggest  that  while  background 
factors  used  by  selection  committees  can  distinguish  between  levels 
of  success  in  an  AEGD  program,  other  factors  should  be  considered. 
Consideration  of  scores  from  the  National  Board  Exams  and  Regional 
Dental  Licensing  Exams,  in  conjunction  with  traditional  background 
information,  might  give  more  useful  predictors  of  levels  of  success 
in  the  AEGD  program. 


25 


26 


References 


1.  Salley,  J.  (1983) .  Conference  summary.  Journal  of  Dental 

Education .  47.410. 

2.  Barnes,  D.M.  &  Hasler,  J.F.  (1988).  Advanced  General 

Dentistry  Program.  Journal  of  Dental  Education.  52 . 271- 

5. 

3.  Santangelo,  M.  (1987).  Required  postdoctoral  education  in 

general  dentistry:  accreditation  issues.  Journal  of 
Dental  Education.  51.280-6. 

4.  Gray,  C.F.  (1987).  Discussion  Summary.  Journal  of  Dental 

Education.  51 . 320-1 ♦ 

5.  Formicola,  A.J.  (1992).  President-Elect's  Address:  A  Call 

to  Action.  Journal  of  Dental  Education.  56:431-4. 

6.  Guion,  R.M. ,  &  Gibson,  W.M.  (1988).  Personnel  Selection  and 

Placement.  Ann.  Rev.  Psvchol..  39:349-74 . 

7.  Wernimont,  P.F.,  &  Campbell,  J.P.  (1968).  Signs,  samples, 

and  criteria.  Journal  of  Applied  Psychology.  52 :  372- 
376. 

8.  Childs,  A.,  &  Klimoski,  R.J.  (1986).  Successfully 

Predicting  career  Success:  An  Application  of  the 
Biographical  Inventory.  Journal  of  Applied  Psychology. 
71,  (1),  3-8. 

9.  Baehr,  M.E.,  &  Williams,  G.B.  (1968).  Prediction  of  sales 

success  from  factorially  determined  dimensions  of 
personal  background  data.  Journal  of  Applied  Psychology. 
65,  662-671. 

10.  Klimoski,  R.J.  (1973) .  A  biographical  data  analysis  of 

career  patterns  in  engineering.  Journal  of  Vocational 
Behavior .  3,  103-113. 

11.  Hough,  L.M.  (1984) .  Development  and  Evaluation  of  the 

"Accomplishment  Record"  Method  of  Selecting  and  Promoting 
Professionals.  Journal  of  Applied  Psychology.  69  (1), 
135-146. 

12.  Campbell,  C.H.,  Ford,  P.,  Rumsey,  M.G.,  Pulakos,  E.D., 

Borman,  W.C.,  Felker,  D.B.,  De  Vera,  M.V. ,  &  Riegehaupt, 
B.J.  (1990).  Development  of  Multiple  Job  Performance 
Measures  in  a  Representative  Sample  of  Jobs.  Personnel 
Psychology.  43 .  (2),  277-300. 


27 


13.  Staat,  R.H.,  &  Yancey,  J.M.  (1982).  The  Admission  Index  in 
the  Dental  School  Admissions  Process.  Journal  of  Dental 
Education.  46  (8),  500-503. 


14.  Potter  R.H.Y.,  &  McDonald  R.E.  (1985).  Use  and  application 

of  structural  models  in  dental  education  research. 

Journal  of  Dental  Education.  49  (3),  145-53. 

15.  Scheetz  J.P.  (1987).  Predicting  Graduation  from  Dental 

School  Using  Admissions  Data.  Journal  of  Dental 
Education.  51  (5),  250-251. 

16.  Westerman  G.H.,  Grandy  T.G.,  Combs  C.G.E.,  &  Turner  C.H. 

(1989) .  Personality  Variables  as  Predictors  of 
Performance  for  First -year  Dental  Sttients.  Journal  of 
Dental  Education.  53  (4),  233-237. 

17.  Simon  J.F.,  &  Chambers  D.W.  (1992).  The  Search  for  a 

Profile  of  Aptitudes  That  Characterize  Dentists.  Journal 
of  Dental  Education.  56  (5),  317-321. 

18.  Wolf  M.C.,  Yablon  P. ,  &  Maykow  K.P.  (1987).  Dentists' 

evaluation  of  their  general  practice  residencies.  Special 
Care  in  Dentistry;  7  (3), 102-5. 

19.  Handelman  S.L.,  Blandford  D.H.,  &  Balzer  J.  (1983).  Impact 

of  general  practice  residency  training  on  dentists  and 
dental  practice.  Journal  of  Dental  Education.  47.(9)  / 
615-22. 

20.  Demby  N.A.,  &  Burakoff  R.P.  (1986).  The  effect  of  general 

practice  residency  training  programs  on  the  eventual  mode 
of  practice  of  graduates.  Special  Care  in  Dentistry; 
6(6),  262-5. 

21.  Weber  R.P.  (1985).  Basic  Content  Analysis.  Sage  University 

Paper  series  on  Quantitative  Applications  in  the  Social 
Sciences,  07-049.  Newbury  Park,  California:  Sage 
Publications. 

22.  Fleiss  J.L.  (1981).  Statistical  Methods  for  Rates  and 

Proportions  (2nd  ed.).  New  York:  John  Wiley  and  Sons, 

pp. 216-18. 

23.  Landis  J.R.,  &  Koch  G.G.  (1977).  The  Measurement  of 

Observer  Agreement  for  Categorical  Data.  Biometrics . 

33,  159-174. 

24.  Afifi  A. A.,  &  Clark  V.  (1987).  Computer-Aided  Multivariate 

Analysis  (4th  ed) .  London:  Wadsworth,  Inc.,  pp. 309-329. 


28 


25.  Aldrich  J.H.,  &  Nelson  F.D.  (1984).  Linear  Probability. 

Loait.  and  Probit  Models.  Sage  University  Paper  series 
on  Quantitative  Applications  in  the  Social  Sciences,  07- 
045.  Newbury  Park,  California:  Sage  Publications. 

26.  Peterson  N.G.,  Hough  L.M.,  Dunnette  M.D.,  Rosse  R.L., 

Houston  J.S.,  &  Toquam  J.L.  (1990).  Project  A: 
Specification  of  the  Predictor  Domain  and  Development  of 
New  Selection/Classification  Tests.  Personnel 
Psychology.  43  (2),  247-276. 

27.  American  Dental  Association,  Commission  on  Dental 

Accreditation  (1988) .  Assessing  educational  outcomes. 
Chicago:  American  Dental  Association. 

28.  Kress  G.C.,  &  Dogon  I.L.  (1981).  A  Correlational  Study  of 

Preadmission  Predictor  Variables  and  Dental  School 
Performance.  Journal  of  Dental  Education.  45  (4) ,  207- 
210. 

29.  Helmreich,  R.L.,  Sawin,  L.L.,  &  Carsrud,  A.L.  (1986).  The 

honeymoon  effect  in  job  performance:  temporal  increases 
in  the  predictive  power  of  achievement  motivation. 
Journal  Applied  Psychology.  71 .  185-88. 


29 


APPENDIX  A 


30 


SEE  PRIVACY  ACT  statement 
ON  DA  FORM  67-8-1 


«  LAST  NAME  ElRST  NAME  MiOOLE  INITIAL 


O*  tHk«  form  Mft  A.A  6113  &«dOOn*<^i 
•9*ncv  tt  uS  A'<Tiy  MiMtVvv  Cant*/ 


^ART  i  ~  AOMtNISTAATIVf  DATA 


UNJT  OACANIZATION.  STATION.  -Z>f*  CODt  0»  APO  MAJOfI  COMMAND 


I  REASON  f^OR  SUBMISSION 


K  COMO 
CODE 


m  NO  Of 
months 


Ytar  fMoPitfil  Pay  i  Yaar  I  Mon<ff  f  Oay 


O  RATED  OFFICER  COFT  iVhtek  ont  and  dale/ 


\  1  GIVEN  TO  OFFICER 

J  ,  - 

:  I  2  FORWARDED  TO  OFFICER 


p  FORWARDING  ADDRESS 


Q  EXPUANATION  OF  NONRAT  ED  PE  R' DOS 


PART  li  '  AUTHENTICATIOM  tkaUd  offtcar  $it^atun>  PART  I  data  and  RATING  OFFICIALS  ONLY/ 


•  NAME  OF  RATER  fLddl.  Fipal.  Mi 


GRADE.  BRANCH.  ORGANIZATION.  DUTY  ASSIGNMENT 


b  NAME  OF  INTERMEDIATE  RATER  iLatl  Fmi  Ml 


GRADE.  BRANCH.  ORGANIZATION.  DUTY  ASSIGNMENT 


I 


NAME  OF  SENIOR  RATER  tLatl  Ftral.  Ml 


GRADE.  BRANCH.  ORGANIZATION  DUTY  ASSIGNMENT 


d.  SIGNATURE  OF  RATED  OFFICER 


•  DATE  ENTERED  ON  I  t  RATED  OFFICER  I  0  SR  MPO  INITIALS  NO  OF 
DA  FORM  2  t  I  MPO  INITIALS  INCL 


PART  III  -  DUTY  DESCRIPTION  iRdirn 


•  PRINCIPAL  DUTY  TITLE 


PART  IV  >  PERFORMANCE  EVALUATION  -  PROFESSIONALISM  iHairr. 


•  PROFESSIONAL  COMPETENCE 


iln  lii’iTiA  I  mrinnih  1^  Im-Iiih'  indn  dtf  thr  ilrgrt-c  «»/  u'4lh  Ihr  Ivlhiwtnji  Alalrmvnl-- 

Oi  ttftnf  dmcnplivv  m(  Ihv  ralrd  ttffu  rr  it  tH  r«*^ii-4  U  d  in  0  brluu  i 


HIGH  DEGREE  LOW  DEGREE 


1.  PoMBiei  capacity  to  acquire  knowiedge/gra«p  concepts 


2.  Demoiiatratca  appropriate  knowledge  and  expertise  in  assigned  tasks 


Z.  MainuiCd  appropriate  level  of  physical  fitness 


4.  Mothrataa.  challanfea  and  develops  subordinates 


5.  Pvrfomu  under  physical  and  mental  stress 


b.  Encourafea  candor  and  frankness  m  subordinates 


7.  Clear  and  concise  in  written  communication 


H  Displays  sound  judgment 


9  Seeks  self-improvement 


10.  Is  adaptable  to  changing  situations 


1 1  Sets  and  enforces  high  standards 


Possesses  military  bearing  and  appearance 


13  Supports  bOfEEO 


J4.  Clear  and  concise  in  oral  communication 


utaplx  iimMtonding  nr  m  *-d* 


1.  DEDICATION 
7  RESPONSIBtLiTY 

3,  LOYALTY 

4.  DISCIPLINE 
S  INTEGRITY 

6.  moral  COURAGE 

7  selflessness 
B  MORAL  STAND¬ 
ARDS 


DA  67  -  8 


REPLACES  DA  FORM  67  7  l  jAN  73  WMiCh  iS  OBSOlE  T(  I  NOV  79 


us  ARMY  OFFICER  EVALUATION  REPORT 


FSAiOO  COV€A(0 


»  MAteO  C'flCtn  S  NAM6 


#AIIT  V  -  AND  rOTtftTlAt  f  VALUATION 


RATED  OFFICER  iS  ASSIGNED  iN  ONfc  Of  MtS  hE  «  06  S-GN  A  Tf  O  SAfc  C‘AC  f  if  S^MOS 


b  AERf  ORMANCE  During  This  RaT^nC  Rf  f  E  R  TO  AA  A  r  >.  •  OA  f  OAM  t>  ■  *  AND  FaRT  <<<  *  c  AhO  t  OA  FORM  El  -  If  t 


□ 


ALWAYS  EXCEEOEO 
REOUiAEVCNTS 


Ow'i>wAi  i  *  I  Ar.t  ^ 
A»  Gw' At  Ml  N?  1 


C  COMMENT  ON  SAf:  Clf  iC  AS^tCTS  Of  TMt  Pf  Af  ORMANCE  Afff,HTO*»ART:,i  OA  f  ORM  AT  .  ■  anO  F  A  R  T  ■ .  •  t,  AN  U  0  A  f  O  AM  «  f  B  ‘  OO  NG^  uS*  *  OR  COMMtHT 
ON  POTENTIAL' 


d  THIS  OFFICER'S  POTENTIAL  FOR  PROMOTION  TOTmENEx^hiGhER  GRADE  'S 


□ 


PROMOTE  AHEAD  Of 
CONTEMPORARIES 


PROMOTE  WiT  H 

contemporaries 


I  I 


OO  NO'  PROMOTf 


1  I  ° 


PART  VII  -  SENIOR  RATER 


a  Potential  evaluation  1^*0  Chcoi^r  t  ar  hj  <  k>s- 

OA 

SR  USS  ONLY 


t 

tt 

AAAAAAAAAAAAi 

fiTHiiiiifff 


FACTORS  ASSOCIATED  WITH  SDCCESS  IN  THE  AEGD 
_ DATA  COLLECTION  FORM _ 

YEAR  GROUP  FY _  ID  NUMBER 

MONTHS  ON  ACTIVE  DUTY  AS  A  DENTIST 

AGE  AT  GRADUATION  FROM  DENTAL  SCHOOL 

NAME  OF  DENTAL  SCHOOL _ 

CLASS  STANDING  IN  DENTAL  SCHOOL  _  OF 

GRADE  POINT  AVERAGE  .  BASED  ON 


PLEASE  CIRCLE  THE  APPROPRIATE  RESPONSE. 
SEX  1  MALE  2  FEMALE 


DENTAL  LICENSE 


1  YES 


2  NO 


PRIOR  SERVICE 


1  OFFICER 


2  ENLISTED  3  NONE 


ROTC  TRAINED 


1  YES 


2  NO 


US  MILITARY  ACADEMY  1  YES 


2  NO 


PROMOTION  TO  MAJOR  1  BELOW  THE  ZONE  2  WITH  PEERS  3  PASSED  OVER 


REGULAR  ARMY 


1  YES 


2  NO 


SELECTED  FOR  SPECIALTY  TRAINING  1  YES 


2  NO 


MILITARY  BASE  FOR  AEGD  TRAINING 


REMARKS 


33 


APPENDIX  B 


34 


AE6D  SUCCESS  INDEX  EVALUATION  FORM 


24  April  1992 


SUBJECT;  Instruction  sheet  for  AEGD  Success  Index  Evaluation  Form 
1.  Guidelines  for  the  Evaluation 

a.  The  "Potential"  category  has  three  areas:  1)  institutional,  2)  dental,  and 
3)  leadership.  Use  the  following  guides  to  help  you  score  these  areas: 


Guidelines 


Scoring 


Institutional 


-  professional  bearing 


-  recommended  advanced  military 

schooling 

-  recommended  for  Voluntary  Indefinite 

or  Regular  Army  Appointment 


4  -  superior 
3  -  far  above  peers 
2  -  above  peers 
1  -  with  peers 
0  -  below  peers 


Dentistry 


-recommended  for  specialty  trning 

-recommended  for  Officer  in 

Charge  of  small/medium  clinic 


4  -  superior 
3  -  far  above  peers 
2  -  above  peers 
1  -  with  peers 
0  -  below  peers 


Leadership 


-recommended  for  leadership  positions 

-recommended  for  t  responsibility 
-recommended  for  early  promotion 


4  -  superior 
3  -  far  above  peers 
2  -  above  peers 
1  -  with  peers 
0  -  below  peers 


c.  The  "Residency  Performance"  section  has  two  areas:  1)  an  evaluation  of  how  well 
the  pzogram  requirements  were  met,  and  2)  a  subjective  appraisal  of  how  well  the 
resident  performed  based  on  the  content  of  the  written  narrative.  Use  the  following 
guides  to  help  you  score  these  areas: 

Guidelines  Scoring 


Reouirements 

Met 

Met 

Met 

of  Program 

Met 

Met 

++++  (outstanding) 

+++  (always  exceeds  +) 
++  (always  exceeds) 

+  (usually  exceeds) 


4 

3 

2 

1 

0 


Content 


Appraisal 


Average  (superior) 
tt  Average 

t  Average  ( Above ) 

Average 
1  Average 


4 

3 

2 

1 

0 


2.  You  do  not  have  to  total  the  scores,  merely  enter  a  number  in  each  appropriate 
box. 
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Table  1 

Reliability  Estimates  of  Review  Ratings 
of  AE6D  Program  Dimensions 


Dimension 

Weighted  Kappa 

Percent  Agreement 

Institutional 

Potential 

.55 

.65 

Leadership  Potential 

.72 

.74 

Dental  Professional 
Potential 

.80 

.84 

Academic  Performance 

.72 

.75 

Overall  Residency 
Performance 

.80 

.82 

Table  2 


AEGD  Success  Criterion  Measures 


i 

Dimension 

Description 

Mean 

S.D. 

1.  Institutional  Potential 

Possesses 
institutional 
bearing, recommended 
for  advanced 
inst itut iona 1 
schooling. 

2.38® 

0.80 

2 .  Leadership  Potential 

! 

Recommended  for 
leadership 
positions, 
increased 

responsibility,  and 
promotion  ahead  of 
peers. 

2.30® 

0.91 

3.  Dental  Professional 
Potential 

1 

Recommended  for 
dental  specialty 
training, 
recommended  for 
supervisor  of  small 
or  medium-sized 
dental  clinic. 

2.43® 

0.87 

4.  Academic  Performance 

Ability  to  meet 
program 
requirements , 
proficiency  in 
clinical  and 
didactic 
assignments. 

2.33*^ 

0.86 

5.  Overall  Residency 
Performance 

Content  appraisal 
of  Officer 

Evaluation  Report: 
outstanding, 
excellent,  average, 
or  below  average. 

2.38^ 

0.89 

6.  Overall  Success  Score 

Summation  of  5 
Dimensional  Rating 
Scores 

11.82 

3.94 

*  Ossfcelow  peers,  l=with  peers,  2=above  peers,  3=far  above  peers,  4=8uperior  to  peers 


0=far  below  average,  l=average,  2=above  average,  3=far  above  average,  4=superior 
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Table  3 


Principal  Component  Analysis 
of  Five  Dimensions  of  AE6D  Success 


PRINCIPAL 

COMPONENT 

DIMENSION 

1 

2 

3 

4 

5 

Institutional 

Potential 

.40 

.46 

.56 

.52 

.21 

Leadership 

Potential 

.47 

.55 

-.63 

-.23 

.16 

Dental 

Professional 

Potential 

.43 

-.63 

-.35 

.49 

.23 

Academic 

Performance 

.44 

-.30 

.40 

-.65 

.36 

Overall 

Residency 

Performance 

.49 

-.07 

.08 

-.05 

-.86 

Proportion  of 
Variance 

.83 

.07 

.  04 

.04 

.02 

Table  4 


Potential  Predictor  of  Variables  Examined 
by  Year  Groups 


Variable 

YRGRP  84 
n=33 

YEAR  GROUP 

YRGRP  86 
nz24 

YRGRP  87 
n=33 

GRADUAXXON  AOB  I 

mean 

26.9  years 

29.3 

27.3 

(s.d.) 

(2.75) 

(3.63) 

(2.32) 

min, max 

24,35 

25,37 

24,33 

GENDER  (%) 

-MALE 

30  (91.0) 

21  (87.5) 

30  (91.0) 

-FEMALE 

3  (  9.0) 

3  (12.5) 

3  (  9.0) 

CLASS  RANK 

■ean 

Top  38% 

Top  39% 

Top  38% 

(s.d.) 

15% 

18% 

20% 

■in,aax 

7%, 76% 

7%, 73% 

3%, 73% 

DENTAL  SCHOOL 

GPA 

Bean 

3.04 

3.03 

3.03 

(s.d.) 

0.22 

0.26 

0.37 

Bln,nax 

2.60,3.50 

2. 6, 3. 5 

2. 4, 3. 8 

PRIOR  SERVICE  (%) 

-Officer 

4  (12.0) 

8  (33.0) 

3  (  9.0) 

-Enlisted 

6  (18.0) 

5  (15.0) 

4  (12.0) 

MILITARY  SCHOOLING  (%) 

ROTC 

5  (15.0) 

9  (37.5) 

5  (15.2) 

USMA 

1  (  3.0) 

0  (0) 

0  (0) 

REGULAR  ARMY  (%) 

Appointment 

8  (24.2) 

3  (12.5) 

1  (3.0) 

TRAINING  SITE  (%) 

-Ft .  Carson 

4  (12.1) 

2  (  8.7) 

6  (18.1) 

-Ft.  Campbell 

0  (  0.0) 

0  (  0.0) 

7  (21.2) 

-Ft .  Jackson 

8  (24.2) 

4  (17.4) 

7  (21.2) 

-Ft.  Henning 

6  (18.2) 

3  (13.1) 

5  (15.2) 

-Ft.  Sill 

6  (18.2) 

7  (30.4) 

5  (15.2) 

-Ft .  Riley 

7  (21.2) 

3  (13.0) 

3  (  9.1) 

-Ft .  Bragg 

2  (  6.1) 

4  (17.4) 

0  (  0.0) 

Table  5 

Simple  Correlations  between  Observed  Variables 


1  23456  7  89 

1 .  Graduation 

Age 

1 

1 

-.11 

-.18 

.16 

-.05 

-.21* 

.11 

.06 

.06 

2 .  Regular  Amy 
Appointment 

-.11 

1 

1 

.10 

.18 

.02 

-.06 

-.27* 

.25* 

.25* 

3.  Class  Standing 

-.18 

.10 

1 

1 

.17 

.14 

.44* 

.02 

.26* 

.26* 

4.  Military 
Education 

.16 

.18 

.17 

1 

1 

.18 

.13 

.01 

.23* 

.23* 

5 .  Gender 

-.05 

.02 

.14 

.18 

1 

1 

.10 

-.01 

-.01 

-.01 

6.  GPA 

-.21* 

-.06 

.44* 

.13 

.10 

1 

1 

-.02 

.06 

.06 

7 .  Year  Group 

.11 

-.27* 

.02 

.01 

-.01 

-.02 

1 

1 

-.07 

-.07 

8.  Program  Success 
Score 

.06 

.25* 

.26* 

.23* 

-.01 

.06 

-.07 

1 

1 

.99* 

9 .  Overall  Success 
Score 

.06 

.25* 

.26* 

.23* 

-*01 

• 

.06 

-.07 

.99* 

— 

=  significant  at  the  p  =  .05  level. 

Program  Success  Score  =  First  principle  component  of  5  Dimensions  of  AEGD  Success. 
Overall  Success  Score  *  The  sum  of  the  5  Dimensional  rating  scores. 
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Table  6 


Regression  of  AE6D  Success  Index  Score 
on  Identified  Predictors 
(n=90) 


Parameter 

Standard 

Variable 

Estimate 

Error 

Probability 

INTERCEPT 

1.911 

4.225 

.3261 

OLDER  MILITARY 

-4.894 

2.009 

.0085 

GRADUATION  AGE 

0.318 

0.151 

.0193 

REGULAR  ARMY 
APPOINTMENT 

1.856 

1.180 

.0598 

CLASS 

STANDING 

1.966 

0.864 

.0127 

PRIOR  MILITARY 
EDUCATION 

2.887 

1.171 

.0078 

Model  F  value  = 

4.42,  P  <  .0013, 

r2  =  .2083, 

(two-tailed  test) 

older  Military  *  interaction  term  combining  prior  military  education  with  graduation 

age  greater  than  30  years.  1  if  >  30  years  old  and  prior  military 
education;  0  if  otherwise. 

Graduation  Age  -  Age  at  graduation  from  dental  school. 

Regular  Army  =  1  if  regular  army  appointment;  0  if  otherwise. 

Appointment 

Class  Standing  =  1  if  class  rank  is  top  50%;  0  if  class  rank  is  bottom  50%. 

Prior  Military  =  1  if  ROTC  or  OSMA;  0  if  otherwise. 

Education 
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Table  7 


EXAMPLES  OF  VALUES  FOR  AE6D  SUCCESS  INDEX  EQUATION 

MODEL  EQUATION 


OVERALL  SUCCESS 
SCORE 


=  1.911  -  4.894*(OLDER  MILITARY)  +  1 . 856* {REGULAR  ARMY)  + 
2-887*(PRIOR  MILITARY  EDUCATION)  +  . 318* (GRADUATION  AGE) 


+  1.966* (CLASS  STANDING) 


SCORE  95%  OLDER  REGULAR  MILITARY  GRADUATION  CLASS 

Cl*  MILITARY  ARMY  EDUC.  AGE  STANDING 

9.54 

8.07 

11.04 

0 

0 

0 

24 

0 

9.86 

8.59 

11.15 

0 

0 

0 

25 

0 

11.83 

10.21 

13.47 

0 

0 

0 

25 

1 

12.75 

10.34 

15.18 

0 

0 

1 

25 

0 

15.03 

12.71 

17.38 

0 

0 

1 

26 

1 

16.89 

14.33 

19.46 

0 

1 

1 

26 

1 

12.09 

10.47 

13.73 

0 

0 

0 

32 

0 

12.05 

9.09 

15.03 

1 

0 

1 

32 

1 

95%  Cl  represents  the  upper  and  lower  confidence  intervals  for  the 
predicted  TOTAL  scores. 


RANGE  OF  TOTAL  SCORES  AND  PERCENTILES 

TOTAL  SCORES  PERCENTILE 


0.0  -  9.86 

Bottom 

10.0% 

<  10.10 

25.0% 

<  11.45 

50.0% 

<  12.75 

75.0% 

>  14.64 

Top 

10.0% 
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Table  8 


Classification  of  AE6D  Participants 
Based  on  Probit  Logistic  Regression 


Nvunber  of  Participants 

Classified  as 

Original 

Group 

Percent 

Correct 

Above  Average 
Success 

Below  Average 
Success 

Successful 

68.6 

35 

16 

Less  tban 
Successful 

66.7 

13 

26 

Total 

67.8 

"Above  Average" 
"Below  Average" 


was  defined  as 
was  defined  as 


above  the  Overall 
below  the  Overall 


Success  Score  mean  of 
Success  Score  mean  of 


11.83. 

11.83. 
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